Comparative efficacy and safety of COVID-19 vaccines in phase III trials: a network meta-analysis

Background Over a dozen vaccines are in or have completed phase III trials at an unprecedented speed since the World Health Organization (WHO) declared COVID-19 a pandemic. In this review, we aimed to compare and rank these vaccines indirectly in terms of efficacy and safety using a network meta-analysis. Methods We searched Embase, MEDLINE, and the Cochrane Library for phase III randomized controlled trials (RCTs) from their inception to September 30, 2023. Two investigators independently selected articles, extracted data, and assessed the risk of bias. Outcomes included efficacy in preventing symptomatic severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection and the incidence of serious adverse events (SAEs) according to vaccine type and individual vaccines in adults and elderly individuals. The risk ratio and mean differences were calculated with 95% confidence intervals using a Bayesian network meta-analysis. Results A total of 25 RCTs involving 22 vaccines were included in the study. None of vaccines had a higher incidence of SAEs than the placebo. Inactivated virus vaccines might be the safest, with a surface under the cumulative ranking curve (SUCRA) value of 0.16. BIV1-CovIran showed the highest safety index (SUCRA value: 0.13), followed by BBV152, Soberana, Gam-COVID-Vac, and ZF2001. There were no significant differences among the various types of vaccines regarding the efficacy in preventing symptomatic SARS-CoV-2 infection, although there was a trend toward higher efficacy of the mRNA vaccines (SUCRA value: 0.09). BNT162b2 showed the highest efficacy (SUCRA value: 0.02) among the individual vaccines, followed by mRNA-1273, Abdala, Gam-COVID-Vac, and NVX-CoV2373. BNT162b2 had the highest efficacy (SUCRA value: 0.08) in the elderly population, whereas CVnCoV, CoVLP + AS03, and CoronaVac were not significantly different from the placebo. Conclusions None of the different types of vaccines were significantly superior in terms of efficacy, while mRNA vaccines were significantly inferior in safety to other types. BNT162b2 had the highest efficacy in preventing symptomatic SARS-CoV-2 infection in adults and the elderly, whereas BIV1-CovIran had the lowest incidence of SAEs in adults. Supplementary Information The online version contains supplementary material available at 10.1186/s12879-023-08754-3.


Rationale
3 Describe the rationale for the review in the context of what is already known, including mention of why a network meta-analysis has been conducted.

Objectives 4
Provide an explicit statement of questions being addressed, with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).

Protocol and registration 5
Indicate whether a review protocol exists and if and where it can be accessed (e.g., Web address); and, if available, provide registration information, including registration number.
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale.
Clearly describe eligible treatments included in the treatment network, and note whether any have been clustered or merged into the same node (with justification).

Information sources 7
Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.

4-2
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.

Study selection 9
State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).

Data collection process 10
Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.

Data items 11
List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.

S1
Describe methods used to explore the geometry of the treatment network under study and potential biases related to it.This should include how the evidence base has been graphically summarized for presentation, and what characteristics were compiled and used to describe the evidence base to readers.

Risk of bias within individual studies 12
Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.

Summary measures 13
State the principal summary measures (e.g., risk ratio, difference in means).Also describe the use of additional summary measures assessed, such as treatment rankings and surface under the cumulative ranking curve (SUCRA) values, as well as modified approaches used to present summary findings from meta-analyses.

Planned methods of analysis 14
Describe the methods of handling data and combining results of studies for each network meta-analysis.This should include, but not be limited to: Handling of multigroup trials; Selection of variance structure; Selection of prior distributions in Bayesian analyses; and Assessment of model fit.

S2
Describe the statistical methods used to evaluate the agreement of direct and indirect evidence in the treatment network(s) studied.Describe efforts taken to address its presence when found.

Risk of bias across studies 15
Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).

Additional analyses 16
Describe methods of additional analyses if done, indicating which were prespecified.This may include, but not be limited to, the following: Sensitivity or subgroup analyses; Meta-regression analyses; Alternative formulations of the treatment network; and Use of alternative prior distributions for Bayesian analyses (if applicable).

S3
Provide a network graph of the included studies to enable visualization of the geometry of the treatment network.

S4
Provide a brief overview of characteristics of the treatment network.This may include commentary on the abundance of trials and randomized patients for the different interventions and pairwise comparisons in the network, gaps of evidence in the treatment network, and potential biases reflected by the network structure.

Study characteristics 18
For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.

Risk of bias within studies 19
Present data on risk of bias of each study and, if available, any outcome level assessment.

Results of individual studies 20
For all outcomes considered (benefits or harms), present, for each study: 1) simple summary data for each intervention group, and 2) effect estimates and confidence intervals.Modified approaches may be needed to deal with information from larger networks.

Synthesis of results 21
Present results of each meta-analysis done, including confidence/credible intervals.In larger networks, authors may focus on comparisons versus a particular comparator (e.g., placebo or standard care), with full findings presented in an appendix.League tables and forest plots may be considered to summarize pairwise comparisons.If additional summary measures were explored (such as treatment rankings), these should also be presented.

S5
Describe results from investigations of inconsistency.This may include such information as measures of model fit to compare consistency and inconsistency models, P values from statistical tests, or summary of inconsistency estimates from different parts of the treatment network.

Risk of bias across studies 22
Present results of any assessment of risk of bias across studies for the evidence base being studied.

Results of additional analyses 23
Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression analyses, alternative network geometries studied, alternative choice of prior distributions for Bayesian analyses, and so forth).

DISCUSSION
Summary of evidence 24 Summarize the main findings, including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., health care providers, researchers, and policymakers).
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review level (e.g., incomplete retrieval of identified research, reporting bias).Comment on the validity of the assumptions, such as transitivity and consistency.Comment on any concerns regarding network geometry (e.g., avoidance of certain comparisons).

Conclusions 26
Provide a general interpretation of the results in the context of other evidence, and implications for future research.

FUNDING
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.This should also include information regarding whether funding has been received from manufacturers of treatments in the network and/or whether some of the authors are content experts with professional conflicts of interest that could affect use of treatments in the network.
* Boldface indicates new items to this checklist.† Text in italics indicates wording specific to reporting of network meta-analyses that has been added to guidance from the PRISMA statement.‡ Authors may wish to plan for use of appendices to present all relevant information in full detail for items in this section.

Updated on March 8, 2021
Please leave this space alone as it will be supplemented by the editorial office when needed.